{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Garimpagem de Dados\n",
    "\n",
    "## Aula 4 - Exercídio de Classificação com kNN\n",
    "\n",
    "13/10/2017"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Dataset:** Titanic: Machine Learning from Disaster\n",
    "\n",
    "https://www.kaggle.com/c/titanic/data\n",
    "\n",
    "Partindo da aula passada:\n",
    "\n",
    "1. Atualizar a função que mede a distância euclidiana para o pacote do scikit-learn \n",
    "\n",
    "2. Implementar uma função que selecione os k vizinhos mais próximos (k > 1)\n",
    "\n",
    "3. Implementar uma função que recebe os k vizinhos mais próximos e determinar a classe correta\n",
    "\n",
    "4. Transformar as features categoricas em numéricas (tip: pandas ou scikit-learn)\n",
    "\n",
    "5. Analisar a necessidade de normalizar as features numéricas (tip: pandas ou scikit-learn)\n",
    "\n",
    "6. Selecionar as features baseada na correlação (tip: pandas)\n",
    "\n",
    "7. Separar o dataset em treino (75%) / teste (25%) / validação (10% do treino)\n",
    "\n",
    "4. Execute o classificador para 30 k's pulando de 4 em 4 e apresente todas as acurácias utilizando o dataset de validação (Qual o melhor k?) [plotar um gráfico com os resultados]\n",
    "\n",
    "5. Executar o classificador para o melhor k encontrado utilizando o dataset de teste e apresentar um relatório da precisão (tip: scikit-learn) [plotar um gráfico com os resultados]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn import metrics\n",
    "from sklearn.metrics.pairwise import euclidean_distances\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.preprocessing import  MinMaxScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import classification_report\n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class KNNClassifier(object):\n",
    "    def __init__(self):\n",
    "        self.X_train = None\n",
    "        self.y_train = None\n",
    "\n",
    "    def euc_distance(self, a, b):\n",
    "        return np.linalg.norm(a-b)\n",
    "\n",
    "    def closest(self, row,k):\n",
    "        #calcula as distancias de cada ponto\n",
    "        dists = np.array([self.euc_distance(row, item) for item in self.X_train])\n",
    "        #ordena os labels mais proximos\n",
    "        nei = dists.argsort()[:k]\n",
    "        labels = [y_train[i] for i in nei]\n",
    "        #conta os mais comuns\n",
    "        counts = np.bincount(labels)\n",
    "        #retorna o mais comum\n",
    "        return np.argmax(counts)\n",
    "        \n",
    "    def fit(self, training_data, training_labels):\n",
    "        self.X_train = training_data\n",
    "        self.y_train = training_labels\n",
    "\n",
    "    def predict(self, to_classify,k):\n",
    "        predictions = []\n",
    "        for row in to_classify:\n",
    "            label = self.closest(row,k)\n",
    "            predictions.append(label)\n",
    "        return predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"train.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "substituindo valores categoricos por numericos e adicionando valores em celulas NaN ou vazias"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Sex_code</th>\n",
       "      <th>Cabin_code</th>\n",
       "      <th>Embarked_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>0</td>\n",
       "      <td>768</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>0</td>\n",
       "      <td>509</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>0</td>\n",
       "      <td>742</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>510</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Survived  Pclass   Age  SibSp  Parch     Fare  Sex_code  Cabin_code  \\\n",
       "0         0       3  22.0      1      0   7.2500         1           0   \n",
       "1         1       1  38.0      1      0  71.2833         0         768   \n",
       "2         1       3  26.0      0      0   7.9250         0         509   \n",
       "3         1       1  35.0      1      0  53.1000         0         742   \n",
       "4         0       3  35.0      0      0   8.0500         1         510   \n",
       "\n",
       "   Embarked_code  \n",
       "0              4  \n",
       "1              2  \n",
       "2              4  \n",
       "3              4  \n",
       "4              4  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#substituindo valores e deletando features descartaveis\n",
    "df[\"Age\"] = df.Age.fillna(df[\"Age\"].mean())\n",
    "df[\"Sex_code\"] = LabelEncoder().fit_transform(df[\"Sex\"])\n",
    "df[\"Cabin_code\"] = LabelEncoder().fit_transform(df[\"Cabin\"])\n",
    "df[\"Embarked_code\"] = LabelEncoder().fit_transform(df[\"Embarked\"])\n",
    "\n",
    "#deletando colunas repetidas e sem importancia\n",
    "del df[\"Sex\"]\n",
    "del df[\"Cabin\"]\n",
    "del df[\"Embarked\"]\n",
    "del df[\"Name\"]\n",
    "del df[\"Ticket\"]\n",
    "del df[\"PassengerId\"]\n",
    "\n",
    "#deletando valores NaN\n",
    "df = df.dropna(axis = 0, how ='any')\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Sex_code</th>\n",
       "      <th>Cabin_code</th>\n",
       "      <th>Embarked_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Survived</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.338481</td>\n",
       "      <td>-0.069809</td>\n",
       "      <td>-0.035322</td>\n",
       "      <td>0.081629</td>\n",
       "      <td>0.257307</td>\n",
       "      <td>-0.543351</td>\n",
       "      <td>0.250409</td>\n",
       "      <td>-0.177347</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pclass</th>\n",
       "      <td>-0.338481</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.331339</td>\n",
       "      <td>0.083081</td>\n",
       "      <td>0.018443</td>\n",
       "      <td>-0.549500</td>\n",
       "      <td>0.131900</td>\n",
       "      <td>-0.500963</td>\n",
       "      <td>0.174782</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>-0.069809</td>\n",
       "      <td>-0.331339</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.232625</td>\n",
       "      <td>-0.179191</td>\n",
       "      <td>0.091566</td>\n",
       "      <td>0.084153</td>\n",
       "      <td>0.133359</td>\n",
       "      <td>-0.042870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SibSp</th>\n",
       "      <td>-0.035322</td>\n",
       "      <td>0.083081</td>\n",
       "      <td>-0.232625</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.414838</td>\n",
       "      <td>0.159651</td>\n",
       "      <td>-0.114631</td>\n",
       "      <td>0.004596</td>\n",
       "      <td>0.071765</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Parch</th>\n",
       "      <td>0.081629</td>\n",
       "      <td>0.018443</td>\n",
       "      <td>-0.179191</td>\n",
       "      <td>0.414838</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.216225</td>\n",
       "      <td>-0.245489</td>\n",
       "      <td>0.014764</td>\n",
       "      <td>0.043784</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare</th>\n",
       "      <td>0.257307</td>\n",
       "      <td>-0.549500</td>\n",
       "      <td>0.091566</td>\n",
       "      <td>0.159651</td>\n",
       "      <td>0.216225</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.182333</td>\n",
       "      <td>0.352913</td>\n",
       "      <td>-0.230493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sex_code</th>\n",
       "      <td>-0.543351</td>\n",
       "      <td>0.131900</td>\n",
       "      <td>0.084153</td>\n",
       "      <td>-0.114631</td>\n",
       "      <td>-0.245489</td>\n",
       "      <td>-0.182333</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.135848</td>\n",
       "      <td>0.119759</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cabin_code</th>\n",
       "      <td>0.250409</td>\n",
       "      <td>-0.500963</td>\n",
       "      <td>0.133359</td>\n",
       "      <td>0.004596</td>\n",
       "      <td>0.014764</td>\n",
       "      <td>0.352913</td>\n",
       "      <td>-0.135848</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.140466</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked_code</th>\n",
       "      <td>-0.177347</td>\n",
       "      <td>0.174782</td>\n",
       "      <td>-0.042870</td>\n",
       "      <td>0.071765</td>\n",
       "      <td>0.043784</td>\n",
       "      <td>-0.230493</td>\n",
       "      <td>0.119759</td>\n",
       "      <td>-0.140466</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Survived    Pclass       Age     SibSp     Parch      Fare  \\\n",
       "Survived       1.000000 -0.338481 -0.069809 -0.035322  0.081629  0.257307   \n",
       "Pclass        -0.338481  1.000000 -0.331339  0.083081  0.018443 -0.549500   \n",
       "Age           -0.069809 -0.331339  1.000000 -0.232625 -0.179191  0.091566   \n",
       "SibSp         -0.035322  0.083081 -0.232625  1.000000  0.414838  0.159651   \n",
       "Parch          0.081629  0.018443 -0.179191  0.414838  1.000000  0.216225   \n",
       "Fare           0.257307 -0.549500  0.091566  0.159651  0.216225  1.000000   \n",
       "Sex_code      -0.543351  0.131900  0.084153 -0.114631 -0.245489 -0.182333   \n",
       "Cabin_code     0.250409 -0.500963  0.133359  0.004596  0.014764  0.352913   \n",
       "Embarked_code -0.177347  0.174782 -0.042870  0.071765  0.043784 -0.230493   \n",
       "\n",
       "               Sex_code  Cabin_code  Embarked_code  \n",
       "Survived      -0.543351    0.250409      -0.177347  \n",
       "Pclass         0.131900   -0.500963       0.174782  \n",
       "Age            0.084153    0.133359      -0.042870  \n",
       "SibSp         -0.114631    0.004596       0.071765  \n",
       "Parch         -0.245489    0.014764       0.043784  \n",
       "Fare          -0.182333    0.352913      -0.230493  \n",
       "Sex_code       1.000000   -0.135848       0.119759  \n",
       "Cabin_code    -0.135848    1.000000      -0.140466  \n",
       "Embarked_code  0.119759   -0.140466       1.000000  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#matriz de correlacao\n",
    "df.corr()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Sex_code</th>\n",
       "      <th>Cabin_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>0</td>\n",
       "      <td>768</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>0</td>\n",
       "      <td>509</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>0</td>\n",
       "      <td>742</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>510</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Survived  Pclass     Fare  Sex_code  Cabin_code\n",
       "0         0       3   7.2500         1           0\n",
       "1         1       1  71.2833         0         768\n",
       "2         1       3   7.9250         0         509\n",
       "3         1       1  53.1000         0         742\n",
       "4         0       3   8.0500         1         510"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#em relacao a survived, temos forte correlacao com as classes pclass, fare, sex e cabin\n",
    "#logo todas as outras serao deletadas\n",
    "del df[\"Age\"]\n",
    "del df[\"SibSp\"]\n",
    "del df[\"Parch\"]\n",
    "del df[\"Embarked_code\"]\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#dividindo entre X e Y\n",
    "Y_train = df[\"Survived\"]\n",
    "X_train = df.loc[:,\"Pclass\":]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.5         0.02049464  0.          0.03841537]\n",
      " [ 1.          0.01517579  0.          0.78991597]\n",
      " [ 0.5         0.04489301  0.          0.41656663]\n",
      " ..., \n",
      " [ 1.          0.01473662  0.          0.54741897]\n",
      " [ 0.          0.05797054  1.          0.88955582]\n",
      " [ 0.5         0.          1.          0.18607443]]\n"
     ]
    }
   ],
   "source": [
    "#normalizando dados e splitando \n",
    "X_train = MinMaxScaler().fit_transform(X_train)\n",
    "Y_train = Y_train.values\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_train, Y_train, test_size=0.25)\n",
    "\n",
    "print(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#fazendo cross validation \n",
    "knn = KNNClassifier()\n",
    "acc = []\n",
    "neibourhs = []\n",
    "\n",
    "for k in range(1,30,4):\n",
    "    knn.fit(X_train, y_train)\n",
    "    result = knn.predict(X_test, k)\n",
    "    accuracy = metrics.accuracy_score(result,y_test)\n",
    "    acc.append(accuracy)\n",
    "    neibourhs.append(k)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAENtJREFUeJzt3X+MZWddx/H3x+k2TAt2QSaEna1uTZrFDciumVQEA0SU\n3aLY0j9I12iw/9QmFKsxK13/ESUG4wLBhMamShUjtiF0LdUQF1SCyh/YWXZxuy2rm+VHO1vpkGZV\nyCbtbr/+MWfM7XR25u7M3d6593m/ksnOec5zpt8nz53P3D7nnHtSVUiS2vEDwy5AkvTiMvglqTEG\nvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9Jjbls2AUs55WvfGVt27Zt2GVI0sg4fPjwd6tq\nqp++GzL4t23bxuzs7LDLkKSRkeRb/fZ1qUeSGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY\n/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEv\nSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLU\nGINfkhpj8EtSYy4bdgEtevDIHAcOneD0mbNs2TzJvt3buXHX9LDLktSIvt7xJ9mT5ESSk0nuXGb/\nVUn+NsnXkhxPckvXfnWSLyZ5tGu/Y9ADGDUPHplj/8FjzJ05SwFzZ86y/+AxHjwyN+zSJDVi1eBP\nMgHcBVwP7AD2JtmxpNt7gUer6vXAW4GPJLkcOAf8VlXtAN4AvHeZY5ty4NAJzj57/nltZ589z4FD\nJ4ZUkaTW9POO/zrgZFWdqqpngPuBG5b0KeBlSQK8FHgaOFdVT1bVVwGq6n+Bx4Cm1zROnzl7Ue2S\nNGj9BP808HjP9hO8MLw/DvwYcBo4BtxRVc/1dkiyDdgFfGW5/0iSW5PMJpmdn5/vq/hRtGXz5EW1\nS9KgDeqqnt3AUWALsBP4eJIfXNyZ5KXAA8BvVNX/LPcDquqeqpqpqpmpqakBlbXx7Nu9nclNE89r\nm9w0wb7d24dUkaTW9BP8c8DVPdtbu7ZetwAHa8FJ4BvAawCSbGIh9D9VVQfXX/Jou3HXNB+66XVM\nb54kwPTmST500+u8qkfSi6afyzkfBq5Ncg0LgX8z8EtL+nwbeBvwL0leBWwHTnVr/p8AHquqjw6u\n7NF2465pg17S0Kz6jr+qzgG3A4dYODn76ao6nuS2JLd13T4IvDHJMeAfgfdX1XeBNwG/AvxMkqPd\n1zsuyUgkSX3p6wauqvoc8LklbXf3fH8aePsyx/0rkHXWKEkaID+yQZIaY/BLUmMMfklqjMEvSY0x\n+CWpMQa/JDXG4Jekxhj8ktQYg1+SGuOjFyWNJR9xemEGv6Sxs/iI08Wn3S0+4hQw/HGpR9IY8hGn\nKzP4JY0dH3G6MoNf0tjxEacrM/gljR0fcboyT+5KGjuLJ3C9qmd5Br+kseQjTi/MpR5Jaozv+DVW\nRu2mnVGrV5fGi/06MPg1Nkbtpp1Rq1eXxjBeBy71aGyM2k07o1avLo1hvA4Mfo2NUbtpZ9Tq1aUx\njNeBwa+xMWo37Yxavbo0hvE6MPg1Nkbtpp1Rq1eXxjBeB57c1dgYtZt2Rq1eXRrDeB2kqi7ZD1+r\nmZmZmp2dHXYZkjQykhyuqpl++rrUI0mNcalHUl+82Wx8GPySVuXNZuPFpR5Jq/Jms/Fi8EtalTeb\njReDX9KqvNlsvPQV/En2JDmR5GSSO5fZf1WSv03ytSTHk9zS77GSNj5vNhsvqwZ/kgngLuB6YAew\nN8mOJd3eCzxaVa8H3gp8JMnlfR4raYO7cdc0H7rpdUxvniTA9OZJPnTT6zyxO6L6uarnOuBkVZ0C\nSHI/cAPwaE+fAl6WJMBLgaeBc8BP9nGspBHgE63GRz9LPdPA4z3bT3RtvT4O/BhwGjgG3FFVz/V5\nrCTpRTSok7u7gaPAFmAn8PEkP3gxPyDJrUlmk8zOz88PqCxJ0lL9BP8ccHXP9taurdctwMFacBL4\nBvCaPo8FoKruqaqZqpqZmprqt35J0kXqJ/gfBq5Nck2Sy4GbgYeW9Pk28DaAJK8CtgOn+jxWkvQi\nWvXkblWdS3I7cAiYAO6tquNJbuv23w18EPiLJMeAAO+vqu8CLHfspRmKJKkffiyzJI0BP5ZZknRB\nBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/JDXG4Jekxhj8ktQYg1+SGmPw\nS1Jj+nnYuhr34JE5Dhw6wekzZ9myeZJ9u7f70G1phBn8WtGDR+bYf/AYZ589D8DcmbPsP3gMwPCX\nRpRLPVrRgUMn/j/0F5199jwHDp0YUkWS1svg14pOnzl7Ue2SNj6DXyvasnnyotolbXwGv1a0b/d2\nJjdNPK9tctME+3ZvH1JFktbLk7ta0eIJXK/qkcaHwa9V3bhr2qCXxohLPZLUGINfkhpj8EtSYwx+\nSWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmP6Cv4ke5KcSHIyyZ3L7N+X5Gj39UiS\n80le0e37zSTHu/b7krxk0IOQJPVv1eBPMgHcBVwP7AD2JtnR26eqDlTVzqraCewHvlRVTyeZBn4d\nmKmq1wITwM2DHoQkqX/9vOO/DjhZVaeq6hngfuCGFfrvBe7r2b4MmExyGXAFcHqtxUqS1q+f4J8G\nHu/ZfqJre4EkVwB7gAcAqmoO+DDwbeBJ4L+r6vPrKViStD6DPrn7TuDLVfU0QJKXs/B/B9cAW4Ar\nk/zycgcmuTXJbJLZ+fn5AZclSVrUT/DPAVf3bG/t2pZzM89f5vlZ4BtVNV9VzwIHgTcud2BV3VNV\nM1U1MzU11UdZkqS16Cf4HwauTXJNkstZCPeHlnZKchXwFuCzPc3fBt6Q5IokAd4GPLb+siVJa7Xq\noxer6lyS24FDLFyVc29VHU9yW7f/7q7ru4DPV9X3e479SpLPAF8FzgFHgHsGPAZJ0kVIVQ27hheY\nmZmp2dnZYZchSSMjyeGqmumnr3fuSlJjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj\n8EtSYwx+SWqMwS9JjTH4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqjMEvSY0x+CWpMQa/\nJDXG4Jekxhj8ktQYg1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtS\nY/oK/iR7kpxIcjLJncvs35fkaPf1SJLzSV7R7duc5DNJvp7ksSQ/NehBSJL6t2rwJ5kA7gKuB3YA\ne5Ps6O1TVQeqamdV7QT2A1+qqqe73X8M/H1VvQZ4PfDYIAcgSbo4/bzjvw44WVWnquoZ4H7ghhX6\n7wXuA0hyFfBm4BMAVfVMVZ1ZX8mSpPXoJ/ingcd7tp/o2l4gyRXAHuCBrukaYB748yRHkvxZkisv\ncOytSWaTzM7Pz/c9AEnSxRn0yd13Al/uWea5DPgJ4E+qahfwfeAF5wgAquqeqpqpqpmpqakBlyVJ\nWtRP8M8BV/dsb+3alnMz3TJP5wngiar6Srf9GRb+EEiShqSf4H8YuDbJNUkuZyHcH1raqVvPfwvw\n2cW2qvov4PEk27umtwGPrrtqSdKaXbZah6o6l+R24BAwAdxbVceT3Nbtv7vr+i7g81X1/SU/4n3A\np7o/GqeAWwZWvSTpoqWqhl3DC8zMzNTs7Oywy5CkkZHkcFXN9NPXO3clqTEGvyQ1xuCXpMYY/JLU\nGINfkhqz6uWco+LBI3McOHSC02fOsmXzJPt2b+fGXct+soQkNW0sgv/BI3PsP3iMs8+eB2DuzFn2\nHzwGYPhL0hJjsdRz4NCJ/w/9RWefPc+BQyeGVJEkbVxjEfynz5y9qHZJatlYBP+WzZMX1S5JLRuL\n4N+3ezuTmyae1za5aYJ9u7df4AhJatdYnNxdPIHrVT2StLqxCH5YCH+DXpJWNxZLPZKk/hn8ktQY\ng1+SGmPwS1JjDH5JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUGINfkhpj8EtSYwx+SWqMwS9JjTH4\nJakxBr8kNcbgl6TGGPyS1BiDX5Ia01fwJ9mT5ESSk0nuXGb/viRHu69HkpxP8oqe/RNJjiT5u0EW\nL0m6eKsGf5IJ4C7gemAHsDfJjt4+VXWgqnZW1U5gP/Clqnq6p8sdwGODK1uStFb9vOO/DjhZVaeq\n6hngfuCGFfrvBe5b3EiyFfh54M/WU6gkaTD6Cf5p4PGe7Se6thdIcgWwB3igp/ljwG8Dz62xRknS\nAA365O47gS8vLvMk+QXgqao6vNqBSW5NMptkdn5+fsBlSZIW9RP8c8DVPdtbu7bl3EzPMg/wJuAX\nk3yThSWin0nyV8sdWFX3VNVMVc1MTU31UZYkaS36Cf6HgWuTXJPkchbC/aGlnZJcBbwF+OxiW1Xt\nr6qtVbWtO+6fquqXB1K5JGlNLlutQ1WdS3I7cAiYAO6tquNJbuv23911fRfw+ar6/iWrVpK0bqmq\nYdfwAjMzMzU7OzvsMiRpZCQ5XFUz/fT1zl1JaozBL0mNMfglqTEGvyQ1xuCXpMYY/JLUmA15OWeS\neeBbPU2vBL47pHIupXEdF4zv2BzX6BnXsS0d149UVV8fe7Ahg3+pJLP9Xp86SsZ1XDC+Y3Nco2dc\nx7aecbnUI0mNMfglqTGjEvz3DLuAS2RcxwXjOzbHNXrGdWxrHtdIrPFLkgZnVN7xS5IGZEMHf5I9\nSU4kOZnkzmHXM0hJvpnkWJKjSUb2o0iT3JvkqSSP9LS9IskXkvxn9+/Lh1njWl1gbB9IMtfN29Ek\n7xhmjWuR5OokX0zyaJLjSe7o2kd63lYY10jPWZKXJPm3JF/rxvV7Xfua52vDLvUkmQD+A/g5Fp7z\n+zCwt6oeHWphA9I9lWymqkb6+uIkbwa+B/xlVb22a/sj4Omq+sPuD/bLq+r9w6xzLS4wtg8A36uq\nDw+ztvVI8mrg1VX11SQvAw4DNwK/ygjP2wrjejcjPGdJAlxZVd9Lsgn4V+AO4CbWOF8b+R3/dcDJ\nqjpVVc+w8OjGG4Zck5aoqn8Gnl7SfAPwye77T7LwyzdyLjC2kVdVT1bVV7vv/xd4DJhmxOdthXGN\ntFrwvW5zU/dVrGO+NnLwTwOP92w/wRhMYo8C/iHJ4SS3DruYAXtVVT3Zff9fwKuGWcwl8L4k/94t\nBY3UcshSSbYBu4CvMEbztmRcMOJzlmQiyVHgKeALVbWu+drIwT/ufrqqdgLXA+/tlhXGTi2sJW7M\n9cS1+RPgR4GdwJPAR4ZbztoleSnwAPAbVfU/vftGed6WGdfIz1lVne/yYitwXZLXLtl/UfO1kYN/\nDri6Z3tr1zYWqmqu+/cp4G9YWNoaF9/p1lsX112fGnI9A1NV3+l+CZ8D/pQRnbdurfgB4FNVdbBr\nHvl5W25c4zJnAFV1BvgisId1zNdGDv6HgWuTXJPkcuBm4KEh1zQQSa7sTj6R5Erg7cAjKx81Uh4C\n3tN9/x7gs0OsZaAWf9E672IE5607WfgJ4LGq+mjPrpGetwuNa9TnLMlUks3d95MsXPDyddYxXxv2\nqh6A7rKrjwETwL1V9QdDLmkgkvwoC+/yAS4D/npUx5bkPuCtLHxS4HeA3wUeBD4N/DALn7L67qoa\nuZOkFxjbW1lYMijgm8Cv9ayzjoQkPw38C3AMeK5r/h0W1sNHdt5WGNdeRnjOkvw4CydvJ1h4s/7p\nqvr9JD/EGudrQwe/JGnwNvJSjyTpEjD4JakxBr8kNcbgl6TGGPyS1BiDX5IaY/BLUmMMfklqzP8B\nyMbWh+LrVl4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a114e5f90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#plotando grafico de vizinhos x acuracia\n",
    "plt.scatter(neibourhs,acc)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "best number of neighbours: 5 with accuracy of 0.821\n"
     ]
    }
   ],
   "source": [
    "#escolhendo melhor numero de vizinhos respectiva acuracia\n",
    "best_k = neibourhs[np.argmax(acc)]\n",
    "best_acc = max(acc)\n",
    "print (\"best number of neighbours: {:d} with accuracy of {:.3f}\".format(best_k,best_acc))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "    Survived       0.79      0.92      0.85       135\n",
      "Not Survived       0.84      0.64      0.72        88\n",
      "\n",
      " avg / total       0.81      0.81      0.80       223\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#resultados para melhor numero de vizinhos\n",
    "knn.predict(X_test,best_k)\n",
    "print(classification_report(y_test, result, target_names=['Survived', 'Not Survived']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
